Integrating Pointing Gestures into a Spanish-spoken Dialog System for Conversational Service Robots
نویسندگان
چکیده
In this paper we present our work on the integration of human pointing gestures into a spoken dialog system in Spanish for conversational service robots. The dialog system is composed by a dialog manager, an interpreter that guides the spoken dialog and robot actions, in terms of user intentions and relevant environment stimuli associated to the current conversational situation. We demonstrate our approach by developing a tour–guide robot that is able to move around its environment, visually recognize informational posters, and explain sections of the poster selected by the user via pointing gestures. This robot also incorporates simple methods to qualify confidence in its visual outcomes, to inform about its internal state, and to start error–prevention dialogs whenever necessary. Our results show the reliability of the overall approach to model complex multimodal human–robot interactions.
منابع مشابه
Dialogue with Agents in a Dynamic World
We have constructed the discourse component for a multimodal user-interface dialogue system. The larger system acts as a conversational agent engaged in dialogue with the human user. Output originating with the backend application system can take the eventual form of spoken English utterances, printed English text, or changes in the graphical display screen, e.g., appearance, disappearance, mov...
متن کاملI Feel You: The Design and Evaluation of a Domotic Affect-Sensitive Spoken Conversational Agent
We describe the work on infusion of emotion into a limited-task autonomous spoken conversational agent situated in the domestic environment, using a need-inspired task-independent emotion model (NEMO). In order to demonstrate the generation of affect through the use of the model, we describe the work of integrating it with a natural-language mixed-initiative HiFi-control spoken conversational a...
متن کاملToward Integrating Natural-HRI into Spoken Dialog
This paper summarizes our previous works in modeling non-verbal behaviors for natural human-robot interaction (HRI) and discusses a path for integrating them into spoken dialogs. While some non-verbal behaviors can be considered “optional” elements to be added to a spoken dialog, some non-verbal behaviors substantially require a harmonized plan that simultaneously considers both spoken dialog a...
متن کاملModelling Paralinguistic Conversational Interaction Towards social awareness in spoken human-machine dialogue
Parallel with the orthographic streams of words in conversation are multiple layered epiphenomena, short in duration and with a communicative purpose. These paralinguistic events regulate the interaction flow via gaze, gestures and intonation. This thesis focus on how to compute, model, discover and analyze prosody and it’s applications for spoken dialog systems. Specifically it addresses autom...
متن کاملOlga - a conversational agent with gestures
The Olga project has developed an animated agent interface for information services. The interface combines a graphical interface, spoken dialogue and an animated 3D ‘human-like’ character for multimodal input and output. The interaction is intelligently managed using techniques derived from spoken dialogue but extended for the graphical modality. The Olga agent is innovative in combining a spo...
متن کامل